1. Field of the Invention
The present invention generally relates to microprocessor and multiprocessor architectures and, more particularly, to thread-level speculative processor execution for achieving high performance and speeding up sequential applications.
2. Description of the Prior Art
As increasing numbers of smaller and faster transistors can be integrated on a single chip, new processors are designed to use these transistors effectively to increase performance. The arising challenge is to find the most effective way to put these transistors to use. Currently, most computer designers opt to use the increasing transistor budget to build ever bigger and more complex uniprocessors. Another possibility is to place large amounts of memory on the chip. Alternatively, multiple processor cores can be placed on a single chip. The later approach is called chip multiprocessor (CMP) design.
Performance improvements using a single complex processor is typically achieved by exploiting ILP (instruction level parallelism), i.e., by finding non-dependent instructions in a program sequence which are then executed at the same time. However, the possible performance gain by exploiting IPL is limited due to the finite amount of ILP present in any particular application sequence.
Placing multiple smaller processor cores on a single chip is attractive because a single, simple processor core is less complex to design and verify. This results in less costly and complex verification process as a once verified module—the processor—is repeated multiple times on a chip. Each processor core on a multiprocessor is much smaller than a competitive uniprocessor, minimizing the core design time. In addition, keeping design partitions small—like a single processor core in a CMP—design tools can handle processor complexity much more easily, compared to competitive complex uniprocessors. However, many important existing applications are written for uniprocessors, and it is a non-trivial task to convert uniprocessor applications into multiprocessor ones. For this, sequential programs have to be explicitly broken into threads and synchronized properly. So far, parallelizing compilers have had only limited success at automatically handling these tasks.
Speculative multithreaded processors represent a possible solution of these difficulties offering high potential performance improvement. A speculative multithreaded processor consists logically of replicated processor cores that cooperatively perform the parallel execution of a sequential program. The sequential program is divided into chunks called speculative threads, and these threads are executed on processor cores concurrently and speculatively. This approach for performance improvement by exploiting coarse-grain parallelism in addition or instead of fine-grain parallelism (e.g., ILP) is called thread level speculation (TLS). In the thread level speculation approach, sequential programs are divided into speculative threads which are then executed concurrently on processor cores. Ideally, there are no data and/or control dependences between the threads, but being parts of the same sequential program, speculative threads are both data and control dependent. The data flow between speculative threads in one direction only—from sequentially older threads to younger ones. (Thus, data used in a younger speculative thread can be a result calculated in an older thread.) To ensure that each program executes the same way that it did on a uniprocessor, hardware must track all inherited dependences. When a younger thread in a sequence causes a true dependence violation, the hardware must ensure that the misspeculation is detected, and the misspeculated thread has to re-execute with the correct data.
To support speculation, a multiprocessor architecture for thread level speculation has to fulfill the following requirements: 1) it has to maintain a notion of the relative order of the threads—i.e., know which thread is executed before some other thread in a sequential program; 2) it has to forward data between parallel threads, or predict data; 3) it has to support mechanism for dependency violation detection—to detect if a read operation has occurred too early; 4) it has to safely discard speculative thread once a dependency violation is detected; 5) it has to commit speculative writes in proper order—only after making sure that this thread would have been executed the same way in a sequential execution; and, 6) it has to re-execute the misspeculated threads with proper data.
A goal of using speculative multithreading is to exploit distant parallelism which can reach significant levels as shown by Ebcioglu et al., “Optimizations and Oracle Parallelism with Dynamic Translation”, Micro 32, Haifa, Israel, 1999. Thus, it would be highly desirable to enable general uniprocessor applications to efficiently execute on CMP architectures by providing a simple, effective way to parallelize the applications.
Hardware support for thread-level speculation is promising, because it eliminates the need for programmers to explicitly divide their original program into independent threads. One such scheme is described for the Hydra CMP system in Hammond et al., entitled “The Stanford Hydra CMP”, IEEE Micro Magazine, 2000. Thread-level speculation takes the sequence of instructions run during an existing uniprocessor program and breaks it into a sequenced group of threads that may be run in parallel on a multiprocessor. To ensure that each program executes the same way that it did originally, hardware must track all inter-thread dependencies. When a “later” thread in the sequence causes a true dependence violation by reading data too early, the hardware must ensure that the misspeculated thread—or at least the portion of it following the bad read—re-executes with the proper data. This is a considerably different mechanism from the one used to enforce dependencies on conventional multiprocessors. There, synchronization is inserted so that threads reading data from a different thread will stall until the correct value has been written. This process is complex because it is necessary to determine all possible true dependencies in a program before synchronization points may be inserted. Speculation allows parallelization of a program into threads even without prior knowledge of where true dependencies between threads may occur. All threads simply run in parallel until a true dependency is detected while the program is executing. This greatly simplifies the parallelization of programs because it eliminates the need for human programmers or compilers to statically place synchronization points into programs by hand or at compilation. All places where synchronization would have been required are simply found dynamically when true dependencies actually occur. As a result of this advantage, uniprocessor programs may be obliviously parallelized in a speculative system. While conventional parallel programmers must constantly worry about maintaining program correctness, programmers parallelizing code for a speculative system can focus solely on achieving maximum performance. The speculative hardware will ensure that the parallel code always performs the same computation as the original sequential program. Since parallelization by speculation dynamically finds parallelism among program threads at runtime, it does not need to be as conservative as conventional parallel code. In many programs there are many potential dependencies that may result in a true dependency, but where dependencies rarely if ever actually occur during the execution of the program. A speculative system may attempt to run the threads in parallel anyway, and only back out speculative execution of the later thread if a dependency actually occurs. On the other hand, a system dependent on synchronization must always synchronize at any point where a dependency might occur, based on a static analysis of the program, whether or not the dependency actually ever occurs at runtime.
A number of multiprocessor architectures with support for thread level speculation have been proposed. In several of these architectures, a program is chopped into threads by the compiler during the compilation time, such as in a multiscalar processor as proposed in the reference to G. S. Sohi, et al. entitled “Multiscalar Processors”, 27th International Symposium on Computer Architecture (ISCA-22), 1995, or as in a superthreaded architecture or trace processor. In other approaches, hardware dynamically forms the threads during the run time, such as in the Dynamic Multithreaded Processor and “Clustered Speculative Multithreaded Processors”, International Conference on Supercomputing 1999 by P. Marcuello and A. Gonzales. All of these architectures require significant changes on the processor core or/and on the L1 and/or L2 level caches to support thread level speculation. These changes include at least one of the following: 1) provision of means for registers forwarding between processors; 2) the addition of new fields in one or more caches to distinguish speculative vs. non-speculative values; 3) a modified processor interface to allow communication of speculative values; and 4) a change of speculation status for the processor. By requiring significant changes to the processor core and/or to the memory nest to enable thread level speculation, existing architectures cannot take advantage of the increased performance potential which TLS offers. To support thread level speculation on an existing processor, the processor core would need massive re-design and complete re-verification process. Similarly for the memory nest, re-design and verification effort makes it prohibitive, or at least very expensive, for already existing cores and system.
Maria Jesus Garzaran, Milos Prvulovic, Jose Maria Llaberia, Victor Vinals, Lawrence Rauchwerger, and Josep Torrellas, “Tradeoffs in Buffering Memory State for Thread-Level Speculation in Multiprocessors”, 9th International Symposium on High-Performance Computer Architecture (HPCA), February 2003, provides a survey of methods for state buffering.
Kranich and Christie's “Method and mechanism for speculatively executing threads of instructions”, U.S. Pat. No. 6,574,725, describes a master/slave mechanism for executing subordinate (slave) threads on a second processor under control of a first master processor. This is significantly less flexible than the previously described approach which separates a program into a sequence of threads. The cited patent does not require state buffering, or thread re-start as the model does not support coherent speculative multithreading.
The cited works all require modifications to a microprocessor used in a speculatively multithreading system. Alas, due to the cost of building microprocessors, it is desirable to reuse existing cores. Thus, what is needed is a method and apparatus for proving thread control and state buffering with dependence violation logic working in conjunction with cores having been designed without support for multiprocessing.
It would be highly desirable to provide a system and method which would enable thread level speculative execution on existing processors and memory systems without requiring costly changes to the processor core or existing cache hierarchy.
It would be highly desirable to provide a method and apparatus for supporting thread-level speculation support on an unmodified pre-existing processing core thus enabling low complexity speculative multi-threading by a combination of hardware and software components.